Since the relation of labels in Binary Relevance (BR) is ignored, it is easy to cause the multi-label classifier to output not exist or less emergent labels in training data. The Multi-Label classification algorithm based on Bayesian Model (MLBM) and Markov Multi-Label classification algorithm based on Bayesian Model (MMLBM) were proposed. Firstly, to analyze the shortcomings of BR algorithm, the simulation model was established; considering the value of label should be decided by the attribute confidence and label confidence, MLBM was proposed. Particularly, the attribute confidence was calculated by traditional classification and the label confidence was obtained directly from the training data. Secondly, when MLBM calculated label confidence, it had to consider all the classified labels, thus some of no-relation or weak-relation labels would affect performance of the classifier. To overcome the weakness of MLBM, MMLBM was proposed, which used Markov model to simplify the calculation of label confidence. The theoretical analyses and simulation experiment results demonstrate that, in comparison with BR algorithm, the average classification accuracy of MMLBM increased by 4.8% on emotions dataset, 9.8% on yeast dataset and 7.3% on flags dataset. The experimental results show that MMLBM can effectively improve the classification accuracy when the label cardinality is larger in the training data.